Skip to content

Conversation

@jbaiera
Copy link
Member

@jbaiera jbaiera commented Sep 4, 2025

Adds validation to field path parsing to ensure that no square brackets are used in the path when flexible mode is enabled. We plan to introduce array indexing syntax to flexible mode in the future (a.b[1].c), but only after it is more widely supported in the ingest node APIs. To ensure that users do not make a habit of using this reserved syntax going forward, we are reserving it to ensure that it remains unused until it is supported.

@jbaiera jbaiera added >non-issue :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP v9.2.0 labels Sep 4, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Sep 4, 2025
@jbaiera jbaiera force-pushed the streams-reserve-square-brackets branch from 10454cf to 646405a Compare September 5, 2025 06:24
* @param pathParts The tokenized field path to parse
* @return An array of Strings
*/
private static String[] parseFlexibleFields(String fullPath, String[] pathParts) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it meaningful that this is named parseFlexibleFields, when the analogous method for classic mode is validateClassicFields? They're both just validating, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I heavily adapted this code from #133790 with the hope that I can rebase it on top of these changes with minimal heartburn. I think this is just the method name from that PR.

Copy link
Member

@masseyke masseyke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few minor questions, but LGTM.

@jbaiera jbaiera added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Sep 12, 2025
@elasticsearchmachine elasticsearchmachine merged commit 6884dbb into elastic:main Sep 12, 2025
34 checks passed
@jbaiera jbaiera deleted the streams-reserve-square-brackets branch September 12, 2025 08:37
gmjehovich pushed a commit to gmjehovich/elasticsearch that referenced this pull request Sep 18, 2025
…ss pattern (elastic#134172)

Adds validation to field path parsing to ensure that no square brackets
are used in the path when flexible mode is enabled. We plan to introduce
array indexing syntax to flexible mode in the future (`a.b[1].c`), but
only after it is more widely supported in the ingest node APIs. To
ensure that users do not make a habit of using this reserved syntax
going forward, we are reserving it to ensure that it remains unused
until it is supported.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP >non-issue Team:Data Management Meta label for data/management team v9.2.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants